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We report a Dalitz-plot analysis of the charmless hadronic decays of neutral B mesons to K -k^ti^ . 
With a sample of (231.8±2.6) x 10''r(4S') -^ BB decays collected by the BABAR detector at the PEP- 
II asymmetric-energy B Factory at SLAC, we measure the magnitudes and phases of the intermediate 
resonant and nonresonant amplitudes for 5° and S° decays and determine the corresponding CP- 
averaged branching fractions and charge asymmetries. The inclusive branching fraction and CP- 
violating charge asymmetry are measured to be B{B^ — > K'^n~Tv°) = (35.7li| ± 2.2) x 10~^, 
and Acp = — O.OSOlg Q5^ ± 0.055 where the first errors are statistical and the second systematic. 
We observe the decay B" -^ K*''(892)n° with the branching fraction B{B° -^ K*°{892)n°) = 
(S.B^Qg ±0.4) X 10~®. This measurement differs from zero by 5.6 standard deviations (including the 
systematic uncertainties). The selected sample also contains B'^ -^ D'V decays where D" — > K^-k~ , 
and we measure B{B° -^ D°tv°) = (2.93 ± 0.17 ± 0.18) x 10"''. 

PACS numbers: 13.66.Bc, 13.25.-k, 13.25.Hw, 13.25.Gv, 13.25.Jx 
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I. INTRODUCTION 

Amplitude analyses of three-body decays of B mesons 
with no charm particle in the final state are well suited to 
study the Cabibbo-Kobayashi-Maskawa (CKM) frame- 
work pj for charged current weak interactions. In the 
analysis of a Dalitz plot the strong phase motions along 
the lineshapes of interfering resonances are measured and 
can be used to constrain the weak phases related to the 
CKM parameters which, in the Standard Model govern 
CP- violation. Following the path [2,l3,S] of the 3-pion B 
meson decays which give constraints on the CKM angle 
"CKM = arg(-VtdVj^/KdK*b)' it has been shown in ^^ 
that B decays into a kaon and two pions are sensitive to 
the angle 7ckm = arg(-KdV;*^/KdV;;). 

In this paper we present a time independent amplitude 
analysis of the flavor-specific B'^ — > K'^tt~-k'^ decay [75 
which finalizes preliminary studies [8|, |9|. The analy- 
sis compares the Dalitz plots of the B^ and B^ decays 
where low-mass Kt: and tttt resonances interfere. Previ- 
ous measurements of the three-body final state [10, lil| 
and subdecays [i3, 113 to a vector and a pseudoscalar 
meson have been published. Other B -^ Ktttt decays 
have been studied in [ij, [T^ [Tfl ^^ . A phenomenologi- 
cal study of three-body B meson decays without charm 
in the final state is presented in [18| . 

This paper is organized as follows. We first present 
in Section |TT] the decay model based on an isobar expan- 
sion of the three-body decay amplitude. The complex 
coefficients of the expansion are the unknowns we seek 
to determine by means of an unbinned extended maxi- 
mum likelihood fit. We describe the detector and dataset 
in Section [mi the procedure used to select the data sam- 
ple in Section [TVl and the fit method in Section |V| The 
results are then described in Section IVII together with 
the accounting of the systematic uncertainties (in Sec- 
tion IVII[) . Finally in Section IVIIIl we summarize our 
findings and present an interpretation of our results. 



II. DECAY AMPLITUDES 

The i3° -^ K^TT^TT^ decay amplitude is a function of 
two independent kinematic variables commonly chosen 
to be the invariant masses squared^, x — m^^^^ and 
y = m^±^o- The Dalitz plot (DP) is the x, y bidimen- 
sional distribution. It is customary to express the decay 
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amplitude as a sum over intermediate (isobar) states: 

Aix,y) = Y,t,e^'f'^f,ix,y), (1) 

j 

and similarly for the B'^ -^ K^n^n'^ Dalitz plot, 

:4(x,y)-^t,e*^.7,(x,2/). (2) 

j 

The isobar coefficients tjc'^'^^ are constant over the Dalitz 
plot. For each decay channel, the isobar phase (f)j is the 
sum of a strong phase, the same for B^ and B° decays, 
and a weak phase which changes sign. The decay dynam- 
ics of an intermediate state are described by the fj{x, y) 
function which structures the Dalitz plot. For instance a 
resonance formed in the K'^'k~ system gives a contribu- 
tion which factorizes as: 



Jj{x,y) = R,{x) X T,{x,y) x WB,{x), 



(3) 



where Rj{x) is the resonance mass distribution or line- 
shape and Tj{x,y) models the angular dependence. 
WBj{x) = ^Bb{Rp*{x)) Bj{Rq(x)), the product of 
Blatt-Weisskopf damping factors [l^, slightly deviates 
from unity as a function of x through the breakup mo- 
menta^ of the (quasi) two body B and resonance decays 
multiplied by a range parameter R. The fj are normal- 
ized, 



\fj{x,y)\'^dx dy = 1. 



(4) 



DP 



We use the Zemach tensor formalism [20, l2l| for the 
angular distribution T- (x,y) of a process by which a 
pseudoscalar B meson produces a spin-J resonance in 
association with a bachelor pseudoscalar meson. For J = 
0, 1, 2, we have: 



T, 



1, 



(0) ^ 



T, 



^'^ - ^[Hp-'i)'-i\p\m% 



(5) 



where'^ p{x, y) [q{x)] is the momentum vector of the bach- 
elor particle (the resonance decay product Q defined be- 
low) measured in the resonance rest frame. For a neu- 
tral (charged) Kir resonance, Q is the pion (kaon), and 
for a dipion resonance, Q is the tt". Notice that these 
choices define for each two-body system the helicity an- 
gle 6j = {pj, (fj) between and tt. 



p* , the momentum of the bachelor particle in the B meson rest 
frame, is equal to the breakup momentum of the studied B meson 
decay. 
^ For simplicity, we have dropped the j index in p and q. 



TABLE I: The nominal model for the decay B° -> A'+tt^tt" 
comprises a nonresonant part and five intermediate states. 
The three types of lineshape are described in the text. The 
resonances masses and widths are from [22|, except for the 
LASS shape [23|. We use the same LASS parameters for 
both neutral and charged Kn systems. Additional resonances 
that may contribute are included in extended models which 
we study to estimate the systematic uncertainties. 



Intermediate state Lineshape 



Parameters 



A. LINESHAPES 

1. The relativistie Breit-Wigner distribution 

The relativistie Breit-Wigner (RBW) parameterization 
is used for i\:*(892)+'0, K:^{IA?,Q)+^°, and K* {1&%Q)+ ^° : 



R^'\x■m„T''^ 



Nominal model 



— X — irrijT ,' [x) 



(6) 



lonresonant 


Constant 


p'(770) 


GS 


ii'*+(892) 
;^*°(892) 


RBW 
RBW 




LASS 



m° = 1415 ± 3 MeV/c^ 

r" = 300 ± 6 MeV 

:utoff m™"^ = 2000 MeV/c^ 

a = 2.07 ±0.10 (GeV/c)-i 

r = 3.32 ±0.34 (GeV/c)-^ 





Additional 


resonances 




p(1450) 


GS 


m = 1439 


MeV/c^ 






r" = 550 


MeV 


p(1700) 


GS 


m = 1795 


MeV/c^ 






r" = 278 


MeV 


A'2*(1430)+'O 


RBW 






A'*(1680)+'" 


RBW 







The mass-dependence of the total vifidth T, can be ig- 
nored for high-mass states. For the low-mass states which 
decay only elastically, it is defined by 



TY\x) = r 



q{x) Y'^^ B'^-'\Rq{x)) 



^ ^ \ qivi]) 



B(''){Rq{m^A) 



(7) 



where rrij is the mass of the resonance j, F^ = Tj{m'^j) 
its width, and the barrier factors (squares of the Blatt- 
Weisskopf damping factors ^]) are: 






(8) 



1 



1 + i?2g2 ' 



B 



(2) _ 



1 



9 + 3i?2q2 + Ji4q4 



All range parameters were set to i? = in the analysis 
but we checked that the systematic deviations associated 
with more realistic values taken or interpolated from [2J| 
are below the numerical accuracy we use to quote our 
results. 



2. The Gounaris-Sakurai distribution 

The Gounaris-Sakurai (GS) parameterization [2£ 
[Eq. dH)] is used for p-(770), p-(1450) and p-(1700): 



Our nominal model (Table |T|) for the decay B" — > 
_ftr+7r~7r° includes a nonresonant contribution which is 
uniformly distributed over the Dalitz plot, and five res- 
onant intermediate states: p"(770), K*{892)+-^° and 
(A'7r)Q . The latter notation introduced by the BaBAR 
experiment |14l | denotes phenomenological amplitudes 
describing the neutral and charged (Ktt) S-waves each 
by a coherent superposition of an elastic effective range 
term and a term for the ii'Q(1430) scalar resonance. It 
describes current knowledge on low energy Kt: systems 
with a small number of parameters. Variations to the 
nominal model are used to estimate the model-dependent 
systematic uncertainty in the results. Three lineshape 
parameterizations Rj{x) are used. Parameters are taken 
from 22] unless stated otherwise. 



i?f«(x;m„F°) = 



1 + d, rym, 



m?j + fj{x) — X — imjTj{x) ' 



(9) 



with the same a;-dependence of the width as for the RBW. 
The expressions of the constant dj and the function fj (x) 
in terms of rrij and F^ are given in Reference (25i] . 



3. The LASS distribution 



For the Kn S-wave resonances, {KTr)^ ' , which dom- 
inate for rriKTr below ra"^""^ — 2 GeV/c^, an effective- 
range parameterization was suggested J26j to describe the 
slowly increasing phase as a function of the Kn mass. We 
use the parameterization as in the LASS experiment [23| , 
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tuned for B decays: 



i?j^^^^(x;mO,rO,a,r) 



2iSE 



q cot Sb — iq 

J 3 go 



(10) 



[{m° 



J J V^ 90 



where 



COt^R = 



1 1 / X 

aq(x) 2 



(11) 



a is the scattering length, and r the effective range (Ta- 
ble [l|. A LASS isobar coherently comprises the actual 
K^{U30) (82%), the effective range term (57%) and the 
(destructive) interference term (39%). 

In this analysis we use a maximum likelihood fit 
to measure the inclusive branching fraction and CP- 
violation asymmetry, 



B 



A. 



incl — 



CP 



= B{B^ ^ K+n-n") 

J^p[\A{x,y)\^ ~ \Aix,y)\'']dx dy 
J^p[\A{x,y)\^ + \Aix,y)\'']dx dy' 



(12) 
(13) 



as well as the isobar fractions FFj, (CP-averaged over 
S" and i?") and the CP-violation charge asymmetries: 



FFk 



IoA\ike'^'M^'y)\ + \'ike'^''fki^,yW]dx dy 



where m = ^/x and 6 are respectively the invariant mass 
and helicity angle of the K^tt^ system, rrimax = 'ms — 
m^o and TUmin — fnj^+ + m^- are the kinematic limits 
of m. The new variables both range between and 1. 



III. THE BABAR DETECTOR AND DATASET 

The data used in this analysis were collected with the 
BaBAR detector at the PEP-II asymmetric e+e^ stor- 
age rings between October 1999 and July 2004. The 
sample consists of about 210.6 fb~ corresponding to 
Ngg = 231.8 ± 2.6 million BB pairs taken on the peak 
of the T{4:S) resonance (on resonance) and 21.6 fb~ 
recorded at a center-of-mass (CM) energy 40 MeV below 
(off resonance) . 

A detailed description of the BaBAR detector is given 
in [2^. Charged-particle trajectories are measured by 
a five-layer, double-sided silicon vertex tracker (SVT) 
and a 40-layer drift chamber (DCH) immersed in a 
1.5 T magnetic field. Charged-particle identification 
is achieved by combining the information from a ring- 
imaging Cherenkov device with the ionization energy loss 
(dE/dx) measurements from the DCH and SVT. Pho- 
tons are detected in a CsI(Tl) electromagnetic calorime- 
ter (EMC) inside the coil. Muon candidates are identi- 
fied in the instrumented flux return of the solenoid. We 
use GEANT4-based 29] software to simulate the detector 
response and account for the varying beam and environ- 
mental conditions. 



7-2 ,2 

Ak T-k —Lk 
Anji — ^ 



tk +tl 



(14) 



and the isobar phases relative to those of the K*^-!:^ 
channel. In those expressions, tj and tj are the fitted 
magnitudes for the intermediate state j; 0j and 0.- are the 
fitted relative phases. Note that, due to interference, the 
fractions FF}, in general do not add up to unity. Never- 
theless, we define the quasi-two-body branching fraction 
for an intermediate state j as follows: 



incl 



Bi = FF, ■ B 



B. THE SQUARE DALITZ PLOT 



(15) 



The accessible phase space of charmless three-body B 
decays is unusually large. Most contributing resonances 
have masses much lower than the B mass. Hence sig- 
nal events cluster along the Dalitz plot boundaries. This 
is also true for background events. Past experience has 
shown that another set of variables, defining the Square 
Dalitz Plot (SDP) [23| is well suited to such configura- 
tions. It is defined by the mapping: 



dx dy > dm! dO' 

m = — arccos(2 

TT rrimax l^min 



(16) 



1),^' 



TT 



IV. EVENT SELECTION 

A. SIGNAL SELECTION AND BACKGROUND 
REJECTION 

To reconstruct 5° -^ K'^Tr~7r'^ decays, we select two 
charged particles and two photons. The charged parti- 
cle candidates are required to have transverse momenta 
above 100 MeV/c and at least 12 hits in the DCH. They 
must not be identified as electrons or muons or pro- 
tons. A K~^ candidate must be identified as a kaon (with 
a misidentification probability smaller than 4%) and a 
7r~ candidate must not be identified as a kaon (with 
a misidentification probability smaller than 7%). The 
misidentification probabilities are momentum dependent 
and therefore vary across the Dalitz plot. A 7r° candidate 
is built from a pair of photon candidates, each with an en- 
ergy greater than 50 MeV in the laboratory frame (LAB) 
and a lateral energy deposition profile in the EMC consis- 
tent with an electromagnetic shower. The invariant mass 
of a 7r° candidate must satisfy 0.11 < rrij^ < 0.16 GeV/c^, 
a wide enough window to accommodate the variation 
of the resolution across the Dalitz Plot from 4.5 to 8 
MeV/c^. 

At the T{AS) resonance, B mesons are characterized 
by two nearly independent kinematic variables, the beam 
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lution in AE, we introduce the dimensionless quantity: 
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FIG. 1: The standard (a) and square (b) Dalitz plots of the 
selected data sample of 8014 events. The structures are more 
spread out in the square Dalitz plot. The D° — > K'^-k~ nar- 
row band is preserved with the choice made for the m' vari- 
able. 



AE' 



2AE - (A£;„,ax + AE„ 



AK 



AK 



(19) 



where the bounds obtained from simulation, A_Emax = 
0.08-0.0014-m^+^-, A£;min = -0.14-f0.005-m;f+^-, all 
in units of GeV follow the variation of the AE resolution. 
We require |A£;'| < 1. 

Continuum e+e^ —tqq{q^ u,d,s,c) events arc the 
dominant background. To enhance discrimination be- 
tween signal and continuum, we select events by using 
a neural network [30| with an output NN which com- 
bines three discriminating variables: the angles of the 
B momentum and the B thrust axis with respect to 
the e+ beam direction in the CM frame and the dif- 
ference 2L2 — Lq between two variables characterizing 
the energy flow about the B thrust axis. We have 
Ln = J2i Pi ' I '^'^s ^i l"7 where the sum runs over all charged 
and neutral particles in the event (except for those in the 
B candidate) whose momenta pi make angles 9i with the 
B thrust axis. The neural network was trained on off 
resonance data and correctly reconstructed signal Monte 
Carlo events. A data sample with about 4000 fully re- 
constructed B° -^ D*~n+ decays with D*~ -^ D°Tr~ 
and D'^ -^ K'^7T~ is used to validate the shapes of the 
distributions on which the selection procedure is based. 

Between 3% and 17% of the signal events have multi- 
ple reconstructed B candidates (usually two) depending 
on their location in the Dalitz plot. When distinct 7r° 
candidates make acceptable B combinations, we choose 
that with the reconstructed 7r° mass closest to the nom- 
inal value of 0.1349 GeV/c^. When several acceptable 
B combinations can be made of distinct charged particle 
sets, we arbitrarily choose one (in a reproducible fash- 
ion by using the date and time at which the event was 
recorded as a random number). 

There are 8014 events in the data sample after the 
selection. The B meson candidate in each event is mass 
constrained to ensure that the measurement falls within 
the Dalitz plot boundary. The resulting standard and 
square Dalitz plots are shown in Fig.[TJ We now describe 
the composition of the selected data sample. 



energy substituted mass and the energy difference: 



TOES = Jis/2+po-ph)^/E^-pjj, 



AE 



El 



s/2. 



(17) 
(18) 



where E and p are energy and momentum, the subscripts 
and B refer to the e"''e~-beam system and the B can- 
didate respectively; s is the square of the center-of-mass 
energy and the asterisk labels the CM frame. We require 
that TTT-Es > 5.27 GeV/ c^. To avoid a bias in the Dalitz 
plot from the dependence on the tt" energy of the reso- 



B. TRUTH-MATCHED AND 
SELF-CROSS-FEED SIGNAL EVENTS 

Using the Monte Carlo simulation as in [Sj, we distin- 
guish between the correctly reconstructed and the mis- 
reconstructed signal events. A correctly reconstructed 
event where the three particles of the B candidate match 
the generated ones, is called a Truth- Matched (TM) 
event. A misreconstructed signal event contains a B me- 
son which decays to the signal mode, but one or more 
reconstructed particles in the B candidate are not actu- 
ally from the decay of that B. Misreconstructed signal 
is called Self-Cross-Feed (SCF). Misreconstruction is pri- 
marily due to the presence of low momentum pions. Con- 
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sequently the SCF fraction varies across the Dahtz plot as 
shown in Fig. [51 For each point in the Dahtz plot there 
is an efficiency s(rn',9') to reconstruct an event either 
correctly or incorrectly. The quality of the reconstruc- 
tion is poor where the SCF fraction fscF{m', 9'), is high. 
This occurs in the corners of the Dalitz plot where one 
of the final-state particles has a low momentum in the 
LAB frame. These variations can be seen in Fig. [5] and 
in Table [TTl which compares efficiencies and /scf values 
averaged over the Dalitz plot for different B^ -^ K^TT~Tr° 
subdecays"* computed using high statistics Monte Carlo 
samples (4.9 x 10^ i?° -^ K^tt'tt^ events). It is impor- 
tant to keep a high efficiency in the Dalitz plot corners 
where the low-mass vector resonances interfere. Overall 
the total efficiency is close to 15%. The main sources of 
inefficiency are the AE' selection (e « 70 — 80%), the 
kaon identification (e « 69%) and the neural network 
selection (e « 60%). 




0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 

m' 



FIG. 2: Fraction of misreconstructed events {/scf), in the 
square Dalitz plot. This figure includes 4.9 x lO'' signal MC 
events. In white is the SCF-free region. 



TABLE II: Signal selection efficiencies (overall and truth- 
matched) and fraction of misreconstructed events (SCF) for 
different signal modes after full selection. As explained in 
the text, e is the overall efficiency for (TM and SCF) signal 
events. Hence etm = £{1 — /scf). The relative statistical 
precision on these numbers is a few tenths of a percent. 

Decay mode e(%) eTM(%) ,fscF(%) 



ii'*+(892)7r- 


13.9 


10.0 


28.1 


K+p- 


14.3 


10.3 


28.3 


7^*° (892)71" 


15.7 


13.6 


13.3 


Nonresonant 


16.2 


15.2 


6.2 



D° 


n° 


16.4 


15.7 


4.0 


D- 


-K+ 


15.8 


14.6 


7.5 



class of events in the data sample, representing about two 
thirds of its size. 



2. Background from other B decays 

Since there is no restriction on any two-body invari- 
ant mass of the final state particles, large backgrounds 
from other B decays occur. We use high statistics Monte 
Carlo samples to study these backgrounds. Conserva- 
tive assumptions about unknown branching fractions are 
made. For instance when only an upper limit U is known 
for the branching fraction of a decay channel, we use 
C//2 ± {7/2 as an input to the simulation. 

Inclusive and exclusive B decays with or without 
charm are grouped into ten classes to be used in the fit. 
Rates, and topological and kinematical similarities are 
studied to define the classes listed in Table Hill There are 
two classes with inclusive b ^ c decays which are distin- 
guished according to whether a genuine D^ is part of the 
B candidate. We keep the exclusive decays B^ -^ D^tt" 
with 13° -^ K~^Tr~ in the data sample because the co- 
pious yield of approximately 400 well identified events 
helps to control the fit algorithm. 



BACKGROUND 



Continuum background 



THE MAXIMUM LIKELIHOOD FIT 



Although the neural network selection rejects 96% of 
the continuum events, this background is the dominant 



* We choose to classify the B" — > D^ir^, D~ K~^ subdecays as 
background since they proceed via a 6 — > c quark-level transi- 
tion while we are studying charmless processes. 



We perform an unbinned extended maximum likeli- 
hood fit to determine the total S" -^ K^tt~tt^ event 
yield, the magnitudes tj , tj and the phases (j)j , (j)j of the 
complex isobar coefficients of the decay amplitude de- 
fined in Eq. ^ and ^. The fit uses the variables m', 
^'i "Iesj ^E' and NN, to discriminate signal from back- 
ground. 
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TABLE III: The list of B-backgrounds retained for the fit 
(Section |V|. For each channel, we give (anticipating Sec- 
tion |Vl]) either the fitted number of events in the data sample 
if its yield is allowed to vary in the fit procedure or the ex- 
pected number otherwise. 



Class Mode 




Events 





B'-. 


Tf+TT- 


10.4 ± 0.5 fixed 


1 


B"^ 


A'*"(892)7,A'*»1430)7, p+tt" 


95.6 ± 5.0 fixed 


2 


B"^ 


p+p~, K*+{?,m)p-, K''{lGm)p 


10.7 ± 3.8 fixed 


3 


B"^ 


D"{K + 7T-)7V" 


424 ± 25 varied 


4 


B+~ 


^K+TV'' 


17.4 ± 1.5 fixed 


5 


B+- 


> 3-body (mainly K^tv~tt~^) 


119 ±34 fixed 


6 


B ~* 


4-body 


30.3 ± 3.4 fixed 


7 


Generic B -^ charm w/o true D" 


382 ± 49 varied 


8 


Generic B —* charm with true D" 


147 ± 17 varied 


9 


B"^ 


■D-{^7t-tt")K+ 


20.4 ± 7.8 fixed 



A. THE LIKELIHOOD FUNCTION 

The selected on resonance data sample is assumed to 
consist of signal, continuum-background and background 
from other B decays. Accordingly the likelihood function 
of event i is written: 



maximizes the extended likelihood function^: 



N 



C^e-^'-^YlC 



(25) 



where N — 8014 events is the size of the data sample and 



Ntot = Ns^g + Ngg + J2c Ne 



is the expected number 



^ tot ^ * sig 

of events. 

The five-dimensional probability density func- 
tions (PDF) V are the products of the four 
PDFs of the measured discriminating variables 
v = {mES,AE',NN,im',e')}, 



'P = l[vi 



Vk, 



(26) 



fe=i 



The correlations among the measurements are handled 
by building conditional PDFs where appropriate (be- 
tween NN and the Dalitz variables for the continuum, 
and between /S.E' and the Dalitz variables for TM signal 
events). Systematic uncertainties account for the corre- 
lations we neglect. 

A total of 37 parameters are varied in the fit (see Sec- 
tion V.D). A summary of the PDF parametizations is 
given in Table [TVl 



*^i — ^sig,i I f^qq.i i ^ ^ L^ I 



(20) 



where the sum runs over all the B-background classes 
listed in Table lllll All background likelihood functions 
have the same expression: 



1 



Cback,i — Nback-zi'i^ — Qi Aback)'Pl 



back', 



(21) 



where qf is the kaon charge in event i, and Aback is the 
charge asymmetry. 

We consider B° and B° separately to build the signal 
likelihood function. 



B. THE DALITZ PROBABILITY DENSITY 
FUNCTIONS 

Since the decay B'^ -^ K^tt^tt^ is flavor-specific (the 
charge of the kaon identifies the b flavor) , the B^ and B^ 
Dalitz plots are independent. However, because the back- 
grounds are essentially flavor blind, we get a more robust 
procedure by fitting them simultaneously. It is enough 
to describe only the i?° Dalitz plot PDF^. A change from 
AtoZ\ (Eq. Ill and da])) gives the W PDF. 



1. Signal 



'-sig,i '-sig,t ^ '~'sig,f 



(22) 
(23) 



Each part has two terms, one for the TM and one for the 
SCF events: 



'~'stg,i — '-TMj 



'-SCF.i 



N+gi {l-fsCF)^^ 



+fsn,.Vt 



TM,i 



'SCF' SCF,i 



(24) 



The signal Dalitz model has been described in Sec- 
tion ini The free parameters are the magnitudes and 
phases defined in Eq. ^ and ^ for all the intermediate 
states of the signal model given in TableU Since the mea- 
surement is done relative to the K*^{892)tt~ final state, 
the phases of this and the charge conjugate channels are 
fixed to zero. The amplitude of B° -^ /•i:*+(892)7r- is 
also fixed but not that of B° -^ K*-{892)tt+ in order to 
be sensitive to direct CP-violation. 



and similarly for £^- ^ . fscF^ the fraction of SCF-events 



stg,i ' 



averaged over the Dalitz plot, assumed to be the same 
for both fiavors, is discussed below. We denote by N^omp 
the expected number of events for species comp. The fit 



(EIJl and ^, and N^^ 



^ The canceling factors 1/Ntot in Eq. 
Eq. I I25I I required for the likelihood functions to be properly nor- 
malized have been omitted for simplicity. 

® We drop the superscript in V'^ in the following. 
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TABLE IV: Summary of the PDF parameterizations. G=Gaussian, Pl=lst order polynomial, NP=noii-parainetric, and 
CB=Crystal Ball. The notation GG(DP) designates a double Gaussian PDF with parameters which vary over the Dalitz 
plot. The Dalitz plot signal model is described in Section HT] The numbers associated with the B background in the last row 
are the class indices in Table IIIII 



Component 



Toes 



"AF' 



NN 



signal (TM) 

signal (SCF) 

Continuum 

B background (non-D) 

B background (3 and 9) 



CB GG(DP) 

NP G 

Argus PI 

two-dim. NP 
signal-like signal-like 



Dalitz 



NP 


see text 


NP 


see text 


see text 


NP in patches 


NP 


NP 


one NP for both 


see text 



The normalization of the component signal PDFs: 

VTMa « £^(1 - fscF.^)\detJ,\\A,\^ (27) 



VscFa cc s.JscFA\detJ\\A\'' ® Rscf] 



(28) 



is model dependent. J^ is the Jacobian matrix of the 
mapping to the square Dalitz plot. The symbol (g) stands 
for a convolution and the R matrix is described below 
[Eq. p3p ]. The normalization requires the computation 
of the integrals 

1 .1 

dm' d9' e{l^fscF)\detJ\fkf;, (29) 



and 



f dm' f d0' efscF\detJ\M:, 
Jo 



f dm' f d9' e\detj\fkf!, 

JQ 



(30) 



(31) 



where the notations of Eq. ([T]) are used. The integrations 
over the square Dalitz plot are performed numerically. 
The weight 



_ ^-^dm' ^-^de' efscF\detJ\\A\' 
^^^ ^l dm' jl dO' e\detJ\\A\'' 



(32) 



in Eq. ([22)1 ensures that the total signal PDF is nor- 
malized. The PDF normalization depends on the decay 
dynamics and is computed iteratively. In practice the 
computation of f scf rapidly converges to a value which 
we fix after a few exploratory fits. 

Studies in simulation have shown that the experi- 



mental resolutions of 



and 0' need not be intro- 



duced in the TM signal PDF. On the contrary, misre- 
constructed events often incur large migrations, when 
the reconstructed m'^,9'j. are far from the true values 
m'^,9'f. We use the Monte Carlo simulation to com- 
pute a normalized two-dimensional resolution function 
RscF{m'r, 9'r;m't,9'i.), with 

1 .1 

dm'^ d9'^RscF{K.K;m't,9't):^l. (33) 

Jo 

Rscf is convolved with the signal model in the expres- 
sion of Vscf [Eq. 



2. Background 

Except for events coming from exclusive B ^^ D 
decays, all background Dalitz PDF are non-parametric 
smoothed histograms. The continuum distributions are 
extracted from a combination of off resonance data and 
a sideband (5.20 < mES < 5.25 GeV/c^) of the on reso- 
nance data from which the _B-background has been sub- 
tracted. The square Dalitz plot is divided into three re- 
gions where different snroothing parameters are applied 
in order to optimally reproduce the observed wide and 
narrow structures by using a two-dimensional kernel es- 
timation technique [S^l- For m' > 0.8 and all 9', a fine 
grained model is needed to follow the peak from D'^ con- 
tinuum production. For to' < 0.8 there are two different 
wide structures corresponding to the continuum produc- 
tion of the p's for 9' below 0.8 and the charged K*'s 
above. 

The B-background (Table [HJ) Dalitz PDFs are ob- 
tained from the Monte Carlo simulation. For the compo- 
nents which model 6 — > c decays with real D'^ nresons, a 
fine grained binning around the D mass is used to con- 
struct the smoothed histograms. The exclusive signal- 
like components 5° -^ Wtt°, D° -^ K+n- and B° -> 
D-K+, D- -^ TT^TT° are modeled with TM-like and 
SCF-like PDFs. The former are products of a Gaussian 
distribution in niKTr (then transformed into m,') and a 
fifth order polynomial in 9' . The 13° Gaussian mass and 
widths are free parameters in the fit, the TM-like D^ 
helicity polynomial coefficients were obtained via an an- 
cillary fit to data, all other parameters are fixed to their 
value in the Monte Carlo. For the B'^ -^ D^K^ where 
D~ -^ 7r~7r°, the Dalitz Plot PDF is a two-dimensional 
smoothed histogram obtained from the Monte Carlo. 



C. THE OTHER PDFS 

1. Signal 

The Toes PDF for TM-signal events is a Crystal Ball 
function [Slj , the mean and the width of which are free 
parameters in the fit. For SCF-signal events we use a 
non-parametric shape taken from the Monte Carlo simu- 
lation. 
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A£^' is correlated with the Dahtz plot variables for 
TM-signal events. To account for the correlation, we 
choose a double Gaussian PDF the mean and standard 
deviation of which vary linearly with "i^+^_ . These pa- 
rameters (intercept and slope) are taken from the Monte 
Carlo except for a global mean of the double Gaussian 
distribution. A wide single Gaussian distribution with 
mean and standard deviation taken from the Monte Carlo 
is used for the SCF-signal AE' PDF. 

The NN PDFs for TM and SCF events are non- 
parametric distributions taken from the Monte Carlo. 



2. Background 



We use the Argus function [32 



f(z — )ozz\/\ — z'^e ^^ ' 

•^ ^ ,.^ max ^ ^ 



-, max 



(34) 



as the continuum 



PDF. 



Toes i^Ui' . The endpoint in^^ and 
the shape parameter ^ are fixed to 5.2897 GeV/c^ and 
—22 respectively (^ = —22 ± 7 from a fit to off resonance 
data). The Ai?' PDF is a linear polynomial whose slope 
is free to vary in the fit. The shape of the NN distribution 
for continuum is correlated with the event location in the 
Dalitz plot. To account for that effect we use for the 
NN PDF a function that varies with the closest distance 
Adaiitz between the point representing the event and the 
boundary of the standard Dalitz plot, 



7'(7ViV;Adaiitz) 



(l-iV7V)^ 
x{Bo + BiNN 



(35) 



B2NN^). 



The A and B coefficients are linear functions of Adaiitz- 
Their expressions in terms of the a^ parameters that are 
varied in the likelihood fit are: 



• One CP-asymmetry for the continuum events. 

• Four parameters related to narrow particle masses: 
the mass and mass resolution for the B^ and the 

• Two parameters, the global mean (slope), of the 
AE' distribution for the TM-signal (continuum) 
events. 

• Four parameters which account for the correlation 
between the NN output and the event location in 
the Dahtz plot [Eq. ((36l) ]. 

• Twenty-one isobar amplitudes and phases. There 
are 6 intermediate states (5 resonances and a non- 
resonant term) and two Dalitz plots. We fix one ref- 
erence amplitude, that of B° -^ K*'^{892)n~ and 
two phases for the latter and its conjugate. There- 
fore we end up with 11 magnitudes and 10 phases 
to be determined by the fit. 




-2 -1 

log[Lg/(Lg+LB)] 



A 


= ai + a4Adalitz 


Bo 


= Co +Ci Adaiitz, 


Bi 


= as + C2 Adaiitz 


B2 


= a2+ CsAdalitz 



(36) 



We use two-dimensional non-parametric distributions to 
describe the _B-background classes in the (mES: AE') 
plane, except for the exclusive B decays to D mesons 
(classes 3 and 9) in Table Hill for which we use the same 
shapes as for the signal. For each _B-background class the 
NN PDF is a non-parametric distribution taken from the 
Monte Carlo. Classes 3 and 9 share the same NN PDF. 



D. THE FIT PARAMETERS 



The following parameters are varied in the fit: 

• Five yields for signal (Nsig), continuum {Nqq) and 
three B background classes (c=3, 7 and 8 defined 
in Table Uni. 



FIG. 3: Ratio of the TM term jUtm,! [first relevant term 
in Eq. ([22}] over the full likelihood function d [Eq. pOJl ] 
for all events. The data are shown as points with error bars. 
The solid histogram shows the projection of the fit result. 
The dark and grey shaded areas represent the continuum and 
B background, respectively. The light grey region shows the 
SCF contribution. 



VI. RESULTS 

The maximum likelihood fit results in a B^(B'^) — > 
A'^TT^Tr" event yield of Nsig = 1377 ± 70 events, where 
the uncertainty is statistical only. There are 5395 ± 104 
continuum events and 424 ± 25 exclusive B — > D^tt" de- 
cays where D^ -^ K^-k" , (not included in Ngig). The 
remaining 833 B background events are detailed in Ta- 
ble |TTT1 

When the fit is repeated starting from input param- 
eter values randomly chosen within wide ranges of one 
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FIG. 4: (a) tties, (b) AE' and (c) NN distributions after 
restricting ruk to enhance signal. The fractions of accepted 
signal and rejected continuum events are respectively 65% and 
71% (a), 21% and 97% (b), and 49% and 83% (c). The data 
are shown as points with error bars. The solid histogram 
shows the projection of the fit result. The dark and grey 
shaded areas represent the continuum and B background, re- 
spectively. The light grey region shows the SCF contribution. 



order of magnitude above and below the nominal val- 
ues for the amplitudes and within the [0, 27r] interval 
for the phases, we observe convergence toward four so- 
lutions with minimum values of the negative loglikcli- 
hood function (NLL= — log(£)) that are equal within 0.2 



units. There are four degenerate solutions. The event 
yields we quote are the averages of the four solutions 
for which the relative spreads are less than 1%. Monte 
Carlo (m', 9') scatter plots generated according to the 
fitted signal model in the four solutions are barely dis- 
tinguishable. We checked with simulated datasets that 
multiple quasi degenerate solutions are to be expected 
when applying our fitting procedure to samples contain- 
ing as many events as the experiment. 

In the Appendix, the fitted parameters are given for 
the four solutions in Table IIX| together with their corre- 
lation matrix (for solution-I) in Tables |X] and IXII The 
isobar fractions, which are not required to sum to one 
in order to accomodate interference, are not significantly 
different from unity. We observe that the fit fractions and 
the CP asymmetries are consistent within less than three 
standard deviations among the solutions, and indeed 
agree much better for the subdecays to a pseudoscalar 
meson and a vector meson. The phases differ substan- 
tially. The four solutions are actually two solutions for 
the B° phases and two solutions for the B'^ phases. The 
uncertainties and correlation coefficients given in these 
tables are statistical only. They are underestimated be- 
cause the fitting program assumes a parabolic shape for 
the NLL in the vicinity of its minimum. This assumption 
overlooks the fit degeneracy. Before we explain how we 
derive consistent statistical and systematic uncertainties, 
we proceed to describe the goodness of the fit. We post- 
pone the discussion of the physical meaning of the fitted 
signal model to Section IVlIII 

To check the validity of the fits and to study the re- 
sults, we have generated 1000 pseudo experiments with 
as many events as in the data sample using PDFs with 
the fitted parameter values. We check that the NLL of 
the experimental fit falls well within the NLL distribution 
in the pseudo experiments. The goodness of the fits and 
their ability to reproduce the real data are similar for all 
four solutions. We show the results of solution-I in the 
following. We compare the likelihood ratio ruk = '"''^Z' 
(see Eq. (l20t and ((24)) ') in the data and in the pseudo 
experiments and see good agreement (Fig. [3]). The dis- 
tributions of the discriminating variables (ttieSj ^E and 
NN) are shown in Fig. [4] for samples that have been en- 
riched in signal events by restricting ruk (computed with- 
out the plotted variable) to large enough values in order 
to maximize the signal significance. 

Fig. [5] shows the Dalitz plot mass spectra over their 
full range with no restriction on ruk . A zoom of the low- 
mass resonance region (below 1.75 GeV/c^) is shown for 
signal-enriched events in Fig. [Bj More details are shown 
in Fig. [7] (Fig. [8]) which display mx+Tr-, {itt-k+tvo) for 
different intervals of the helicity angles Ox+ti-, {(^k+tt")- 
The interference between the scalar and vector K* is ev- 
ident from the opposite sign of the forward-backward he- 
licity asymmetries below and above the K*{892). This 
effect is seen with sufficient statistics in these plots which 
include both B^ and B^ decays, because the measured 
phase differences are similar in both cases (Table lIXp . 
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FIG. 5: The invariant mass spectra for all events: m^+^- 
(a), rrij^+^o (b) and m^-^o (c). The data are shown as points 
with error bars. The solid histogram shows the projection of 
the fit result. The dark and grey shaded areas represent the 
continuum and B background, respectively. The light grey 
region shows the SCF contribution. The D" mass peak is 
prominent in the top plot. 



FIG. 6: The signal-enriched spectra for masses between 
threshold and 1.75GeV/c^: mjf+^- (a), rrij^+^o (b) and 
m^~^o (c). The fractions of accepted signal and rejected con- 
tinuum events are respectively 51% and 89%. The data are 
shown as points with error bars. The solid histogram shows 
the projection of the fit result. The dark and grey shaded ar- 
eas represent the continuum and B background, respectively. 
The light grey region shows the SCF contribution. 
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FIG. 7: Signal-enriched spectra of m^+^- in the low-mass 
resonance region and different ranges of the helicity angle, 
0^+^- . (a) < eK+-^'- < 90 degrees, (b) 90 < 0^+^- < 180 
degrees. The data sample is enriched in signal events as in 
Fig. [6] An interference between the vector and scalar A'*" 
is apparent through a positive forward-backward asymmetry 
below the K*{892) and a negative one above. The data are 
shown as points with error bars. The solid histogram shows 
the projection of the fit result. The dark and grey shaded ar- 
eas represent the continuum and B background, respectively. 
The light grey region shows the SCF contribution. 



FIG. 8: Signal-enriched spectra of m^+^o in the low-mass 
resonance region and different ranges of the helicity angle, 
Bji+^o. (a) < 6ii+.^o < 90 degrees, (b) 90 < Oj^+^o < 180 
degrees. The data sample is enriched in signal events as in 
Fig. [6l An interference between the vector and scalar K* 
is apparent through a negative forward-backward asymmetry 
below the K*{892) and a positive one above. The data are 
shown as points with error bars. The solid histogram shows 
the projection of the fit result. The dark and grey shaded ar- 
eas represent the continuum and B background, respectively. 
The light grey region shows the SCF contribution. 
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non-parabohc and shaUow because the plotted Ax^ is the 
smallest of those from all solutions. The dashed line scan 
corresponds to the statistical uncertainty. The full line is the 
scan after smearing by the systematic uncertainties described 
in Section rVTil 



We take into account all four solutions of the fit to 
quote the final results of the analysis. To determine the 
statistical uncertainty on a physical parameter p, we fix it 
at successive values spanning its range of interest and re- 
peat the fit, maximizing the projected likelihood function 
(Eq. (j25p where pis frozen). NLL(p), the minimum where 
the fit converges given p, is not always associated with the 
same solution. Therefore the NLL(p) envelope curve is 
far from a parabola. The fiatness of its shape around the 
overall minimum reflects the fit degeneracy. The parame- 
ter values at which Ax^ = 2 [N LL ~ N LLbestfit) reaches 
unity bound the one-standard deviation confidence inter- 
val. The scan of the K*^ {%^2)tt^ isobar fraction is shown 
in Fig. [9l The graph also shows the envelope curve ob- 
tained when the systematic uncertainty is included. The 
null value of the branching fraction is excluded with a 
statistical and systematic significance of 5.6 standard de- 
viations. Thus, this is the first observation of the decay 
_B" — > _ft'*"(892)7r°. Other scans are presented in the Ap- 
pendix. 



VII. SYSTEMATIC UNCERTAINTIES 

Variations around the nominal fit are tried to study 
the dominant systematic effects which are summarized 
in Table IVl For each parameter of interest p {FF, Acp, 
(j)) , the positive (negative) deviations from each effect are 
summed in quadrature to obtain total upward (down- 
ward) systematic errors Sp+ (dp-). The total projected 
likelihood function Ltotip) is computed as the convolu- 
tion of the fit projected likelihood function defined at 
the end of the previous section, by a bifurcated Gaus- 
sian distribution with a lower (upper) standard devia- 
tion 5p^ {5p^). The scan of Ctot drawn as a solid line 



on Fig. [9] (and subsequent ones) is used to determine the 
total confidence interval which accounts for both statis- 
tical and systematic uncertainties. Finally, the upward 
(downward) systematic uncertainty on p (Table IVip is 
the quadratic difference between the upward (downward) 
limits of the total and statistical confidence intervals. 
Note that when p central values significantly differ be- 
tween the four fit solutions, the lowest Pmin and highest 
Pmax are used in the determination of the downward (up- 
ward) uncertainties and an additional systematic uncer- 
tainty ±{pmax -Prrnn)/'^ IS quotcd in Tablc EB 

While simple methods can be used to estimate the 
systematic effects on the isobar fractions and CP- 
asymmetries, it is often necessary to estimate upper lim- 
its for phases. In such cases (the rows labeled Other 
in Table W\i we conservatively use the upper limits at the 
90% confidence level as the systematic uncertainties. 

To estimate the contribution of other reso- 
nances, we fit the on resonance data with extended 
signal models including one e2;ira-resonance on 
top of those in the nominal signal model. The 
p-(1450)X+, p-(1700)X+, X*°(1430)7rO, ii:*^(1430)7r-, 
i(:*°(1680)7rO and ii'*+(1680)7r- final states have been 
studied. These fits are not significantly better than the 
nominal fit. The isobar fractions of all extra resonances 
are below 5%. None is significant. The isobar fractions 
of the nominal resonances change by non significant 
amounts which we record as {Dalitz Plot Model) sys- 
tematic uncertainties in Table [V] Changes in Acp are 
also recorded. As we explain in Section fVIIII our fitting 
procedure is sensitive to some phase differences but not 
others. When there is sensitivity the systematic Dalitz 
Plot Model effects are insignificant. When there is no 
sensitivity dramatic deviations occur. However since 
these phase differences are essentially undetermined by 
the fit we do not record these changes as systematic 
effects. Note that we have not included the low mass k 
resonance which is not established as it is an alternative 
to the {Ktt)q'^ isobar from the model. It has been 
verified however that adding it to the model results 
into a destructive interference with the latter state, 
an insignificant n amplitude and no change in the 
A:*''(892)7r° numeric results. 

There are fixed parameters in the nominal fit model. 
We estimate the associated [Shape parameters, and B 
Background) systematic uncertainties by repeating the 
fit giving the studied parameter values at ±1 standard 
deviation from its fixed value or at the limits of a conser- 
vative range. The fixed parameters to which the isobar 
fractions are sensitive are fscF [Eq- (EH)] and the A£'' 
signal PDF parameters. fscF was varied between 10 
and 16%, a range that is inferred from the comparison 
of the data and simulation oi B —^ Dp events as ex- 
plained in References [3, [3J| . The Acp mainly depend 
on the fixed CP-asymmetries of B background classes 1 
and 7 (see Table|lll|. We vary them by ±0.5 and ±0.2 to 
determine the {B background ) systematic uncertainties. 
These intervals were chosen after inspection of the latest 
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TABLE V; Summary of the systematic uncertainties. The systematic uncertainties are equal for the phases <^ and 0. AU phases 
cj} are referenced to ')f>x*+(892)7r- s-nd cj> to 4>k*-(s92)tt+ ■ The systematic uncertainties associated with the Dahtz plot model and 
the lineshapes are negligible for the phases of the Kir S-waves which are the only phase angles that are measured with some 
statistical accuracy. For the others which are essentially undetermined by the fit we do not quote these undefined systematic 
uncertainties. 



ii'*+(892)7r" 



A'*"(892)7r" 



p(J7Q)-K+ 



NR 



Fraction (%) 



Ac 



Dalitz Plot Model 

Shape Parameters 

B Background 

Lineshapes 

Continuum DP PDF 

Fit Bias 

Total 



±0.32 

+0.31 
-0.34 
+0.06 
-0.14 
+0.06 
-0.04 

±0.31 
±0.23 

+0.59 
-0.62 



Dalitz Plot Model 

Shape Parameters 

B Background 

Lineshapes 

Continuum DP PDF 

Fit Bias 

Other 

Total 



±0.11 

+0.30 
-0.20 
+0.19 
-0.15 
+0.01 
-0.02 

±0.49 
±0.24 



+0.66 
-0.61 



(ATtt),*/ tt" Dalitz Plot Model 

Shape Parameters 

B Background 

Lineshapes 

Continuum DP PDF 

Fit Bias 

Other 

Total 



±1.56 

+ 1.14 
-0.91 
+0.32 
-0.28 
+0.02 
-0.04 
±0.82 

±0.10 



+ 2.12 
-2.00 



(7^71)5%" Dalitz Plot Model 

Shape Parameters 

B Background 

Lineshapes 

Continuum DP PDF 

Fit Bias 

Other 

total 



±2.81 

+ 2.30 
-0.57 
+0.40 
-0.46 
+0.05 
-0.06 

±0.73 
±0.19 



+3.73 
-3.00 



Dalitz Plot Model 

Shape Parameters 

B Background 

Lineshapes 

Continuum DP PDF 

Fit Bias 

Other 

Total 



±0.98 

+0.34 
-0.43 
+0.17 
-0.25 
+0.04 
-0.03 
±0.45 

±0.15 



H.15 
-1.19 



Dalitz Plot Model 

Shape Parameters 

B Background 

Lineshapes 

Continuum DP PDF 

Fit Bias 

Other 

Total 



±0.41 

+0.46 
-0.51 
+0.64 
-0.24 
+0.04 
-0.03 

±0.91 
±0.22 



+ 1.29 

-1.17 



±0.07 
±0.01 
±0.05 
±0.000 

±0.01 

±0.09 



±0.09 

+0.08 
-0.03 
+0.04 
-0.05 

±0.002 



±0.01 



+0.13 
-0.11 



±0.04 
±0.01 
±0.05 
±0.001 

±0.01 

±0.07 



±0.04 
±0.04 

+0.10 
-0.08 

±0.001 
±0.01 



+0.11 
-0.10 



<t> (deg 



±0.03 

+0.01 
-0.02 

±0.02 


- 


±0.001 


- 


±0.01 





±0.04 


- 


±0.04 

+0.03 
-0.02 
+0.07 
-0.08 

±0.001 


undefined 
undefined 




± 6.3 


±0.01 


±16.5 




± 4.1 


±0.09 


±18.2 



± 2.6 
± 9.6 
± 1.5 
±10.1 



± 5.2 
±14.5 
± 4.4 
±16.0 



undefined 



undefined 
± 3.6 
±13.7 
± 2.6 
±14.4 



undefined 



undefined 
± 5.8 
±6.8 
± 2.6 
± 9.3 
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available measurements [35|. 

The variations of the physical parameters of the reso- 
nances in the nominal signal model are recorded as (Line- 
shape) systematic uncertainties. In particular, variations 
of the barrier coefRcients R in [Eq. ([5])] cause no signifi- 
cant effects. 

The method used to determine the continuum square 
Dalitz plot PDF (Section V.B) has been extensively 
tested in the Monte Carlo. The spread of the yields across 
those tests is used to estimate the associated (continuum 
DP PDF) systematic errors. Among the dominant effects 
is the variation of the distortion of the Dalitz plot dis- 
tribution as a function of toes due to the B meson mass 
constraint. 

To estimate the Fit Bias uncertainties stemming from 
the imperfection of the fit model (most importantly the 
too simple description of the SCF but also the neglected 
correlations between toes and AE' for TM signal and 
_B-background events, and the neglected resolution effect 
smearing the K*^ (892) peak), we record the fitted biases 
and spreads in fits performed on large Monte Carlo sam- 
ples with GEANT4 signal events and the actual amount 
of background generated along with their PDFs. Since 
the SCF prevails in the Dalitz plot corners where the res- 
onances interfere, this kind of systematic uncertainty is 
the dominant one for the phase measurements. 

For the B meson branching fraction measurements, we 
have to include additionally the effect of imperfections 
in the event reconstruction on the efficiency. Adding 
in quadrature the uncertainties associated with track- 
ing (1.6%), charged particle identification (2%), tt^ re- 
construction (3%), the efficiency of the selection require- 
ments (0.3% for Toes, 1-2% for AE, 2% for the NN), 
and the integrated luminosity (1.1%), we obtain a global 
systematic percent error of 4.7%. 



VIII. INTERPRETATION 



The final results are given in Tables IVIHVIIll The to- 
tal branching fraction B(B^ -^ K+Tr-n°) = (35.7l?J ± 
2.2) X 10~^ and the global CP-asymmetry Acp — 
-OmOto°oti ± 0-055 are compatible with the published 
measurements from Belle flT| (36.6 tH ± 3.0) x 10"*^ 
and 0.07 ± 0.11 ± 0.01 respectively. The decay^ B° -^ 
K*'^ (892)7t'^ is observed with a significance of 5.6 stan- 
dard deviations (including systematics). We measure 
B(B° -^ A'*"(892)7rO) = (3.6 ± 0.7 ± 0.4) x 10"^, just 
at the edge of the 90% upper limits of previous ex- 
periments (3.5 X 10"*^ by Belle pjj] and 3.6 x lO''^ b; 
CLEO dll) combined to 3.5 x lO'^ in Reference [1 



'' Isospin conservation is assumed for the strong decays of a 1=1/2 
K meson resonance fe* to Kn when we compute the branch- 
ing fraction of the quasi-two-body process B^ -^ k'-K, namely 



1/3. 



The subdecay branching fraction for B^ -^ K*^(892)tt 
agrees with previous measurements from Belle [ll|, [13] 
and BABAR \l^ in both K*+ -^ K+tt^ and A:°7r+ de- 
cay channels. Averaging the BABAR measurements one 
obtains B(B^ -* K*+(m2)TT-) = (11. 7+^;^) x lO^^ and 
Acp(B° -^ K*+(892)n-) = -0.14 ±0.12. The branch- 
ing fraction for B'^ -^ p^K^ is lower than those mea- 
sured by Belle [11] and CLEO [12] but in agreement 
within errors. If we assume that the (Ktt)*q ' isobars are 
pure isospin-1/2 and neglect possible non Kn final states, 
we determine the following effective branching fractions: 
B(B° -^ (KT:)*o+n-) = (28.2tlltti ± 5.2) x 10-^ and 
B(B° ^ (KT:)*o°Tr°) = (IS.ltlitH ± 3.6) x lO^^. A 
consistency check of our analysis is provided by our mea- 
surement of the branching ratio, B(B''^ -^ D'^tt'^) = 
(2.93 ± 0.17 ± 0.18) X lO"'' in agreement with its world 
average and that of the branching fraction of the decay 

There is no evidence of direct CP-violation in any 
resonant subdecay. In Tables |Vl] and IVIIII we give up- 
per limits at the 90% statistical confidence level on the 
branching fractions of resonances that might contribute 
to A'+tt^tt" but are not part of the nominal signal model. 

We also search for direct CP-violation in the inter- 
ference between pairs of isobars (Pi, Rj) by comparing 
the interference patterns in the P° and P° Dalitz plots. 
In the figures described in what follows, we display three 
NLL scans for in turn A(/)y = (pj — cjyi, the phase difference 
between the resonances in P" decays, A0,j- = (j)j — 0^, 
the same for P" decays and d(j)ij = A<Pij — A(f>ij- A 
marked minimum in a A0 (Acjj) scan indicates a siz- 
able interference in P" (P") decays. Evidence for direct 
CP-violation would be seen if (S0„im = A0„jj„ — A(f>„iin 
were significantly away from zero. One standard devia- 
tion confidence intervals (la c.i.) bounds are graphically 
seen as the intersection points between the solid scan 
curves (which incorporate both statistical and system- 
atic uncertainties) and the Ax^ — 1 horizontal line. The 
results are collected in Table IVIII All isobar pairs for 
which the 95% confidence intervals in Scj) are non trivial 
(smaller than ±180 degrees) exhibit sizeable interference 
patterns in P° and P° decays. The Ax^ value for 6(j) = 
corresponds to the square of the direct CP-violation sig- 
nificance in standard deviation units. A scan of the Ri 
and Rj lineshapes with more statistics could enable one 
to disentangle the strong phase motions and determine 
the weak phase. For the scans where interference is elu- 
sive, we quote in Table FVlTl the maximum Ax^ over the 
±180 degrees scanned range. For the Kn systems, we see 
no significant interference pattern between charged and 
neutral P-waves (Fig. [T0|) at the 95% confidence level; 
similarly for the S-waves (Fig. [TT|) . Only a small range 
(A(j) between -131 and -75 degrees) is excluded at the two 
standard deviation level for the P° P-waves. These obser- 
vations are not unexpected for the Ar*(892)7r final states 
since the K* resonances are quite narrow and therefore 
have a small overlap in the Dalitz plot. Furthermore 
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TABLE VI; Final results for rates and CP-asymmetries. The quasi-two-body branching fractions Bj have been computed 
from the isobar fractions FFj using Eq. (|15|) . The statistical uncertainties are given first. The statistical and systematic 
uncertainties are calculated by scanning the NLL close to its minimum taking into account the four fit solutions and recording 
the values where the NLL increases by one unit above its minimum. For the final states with Kn S-waves, a second systematic 
uncertainty covers the spread between the best fit values from the four solutions. For resonances that are considered in extended 
fit models, we quote upper limits at the 90% confidence level (based on statistical uncertainties only) for the isobar fractions 
and the quasi-two-body branching fractions. 



isobar j 



FF, (%) 



B, (10-'') 



^CP 



Js:*+(892)7r- 


ii.sti'gio.e 


4.21°^ ± 0.3 


K*°{892y 


f. 7+1.3+0.7 


2.4 ±0.5 ±0.3 


{Kn)l+n- 


26.3tli1:ll±i-9 


9.4tl-ill:t ± 1.8 


{KnYo^" 


2A.3tl:"et.ll±6.7 


8.7lJ-9tl-3±2.2 


p-{770)K+ 


22.5I3:? ± 1.2 


S.Ottl ± 0.6 


N.R. 


12A±2.6tli 


4.4 ±0.9 ±0.5 


Total 


102.3J;I:J±4.1 


35.7+?;^ ± 2.2 



-0.19: 



±0.04 



-0.09lo:24 ± 0.09 



+0.17^111 ±0.09 ±0.20 
-0.22 ±0.121^1? ±0.27 

+O.Ut.oit ±0.07 
in oo+o.io+o.ii 

-t-U.^O_Q 27_0.10 



-0.030: 



± 0.055 



FFj, [Upper Limits] (%) 



Upper Limits (10 



p-(1450).K'+ 


9+2.2 

^■^-1.5' 


[5.91 


p-(1700).K'+ 


n 7+i'2 

U-'-0.6' 


[3.1] 


Kf{U30)Tr° 


1.2lil 


[3.6] 


A'*+(1430)7r- 


5.2+11 


[7.5] 


K*°{1680y 


3.011 j. 


[5.5] 


K*+{1680)n' 


5.71?:?, 


[8.9] 



2.1 
1.1 
1.3 
2.7 
2.0 
3.2 



TABLE VII: Final results for phases. When there is sensitivity, the results are the one standard deviation confidence interval 
(1 a c.i.) for the phase angle measurements (in degrees). The statistical and systematic uncertainties, determined by the 
NLL scan method described in Fig. |9]are included. The interval bounds can be seen on the graphs referenced in the second 
column, as the intersections of the solid scan curves with the (A^^ = 1) horizontal dashed lines. The Ax^ evaluated for zero Stj) 
measures the significance (squared) for direct CP- violation. When there is no sensitivity we give the maximum Ax^ reached 
over the scanned region. 



Interference pattern 



Graph Resuhs A0 for B" A(t) for B 



A fni- rO 



A(t)-A4> Ax (5(1) = 0) 



K TT neutral— charged P-waves Fig. [10] Ax 
K TV neutral— charged S-waves Fig. [TT] Axl 



2 

MAX 



j'F K^ - A"*±7r'F 



Fig. [14] Axl 



Charged K n P-wave - S-wave Fig. [12] 1 a c.i. 

Neutral K n P-wave - S-wave Fig. [13] 1 a c.i. 

NR — charged S-waves Fig. [15] 1 a c.i. 

NR — neutral S-waves Fig. [16] 1 a c.i. [ 
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FIG. 10: The phase difference between the K* and the 
charged K* for the P-wave A'*(892). The three diagrams are 
the NLL scans for the _B° (a) and S" (b) decays as well as 
their differences (c). The dashed line gives the statistical un- 
certainty, and the solid line, the total uncertainty. The data 
do not indicate preferred angles, except for the [-131, -75] de- 
gree range which is excluded for B" at the two standard de- 
viation level. The four fit solutions find their NLL minimum 
for distinct phases. The vertical scale stops at 2A(NLL)=4 
slightly above 3.84 which is the 95% confidence level. A hor- 
izontal dashed line at 2A(NLL) = 1 shows the one standard 
deviation level. 



FIG. 11: The phase difference between the {Ktt)q and the 
{KTr)Q S-waves. The three diagrams are the NLL scans for 
the B" (a) and B° decays (b) as well as their differences (c). 
The lines are drawn as for Fig. IIUI The data do not indi- 
cate preferred angles. The four fit solutions find their NLL 
minimum for distinct phases. 



the coherent sum of amplitudes that interfere might be- 
have like the model in [^, [y] with a single weak phase 
(equal to 7cKM in the absence of electroweak penguin 
diagrams). In such a scheme, one weak phase would be 
missing to enable direct CP-violation to take place. In 
contrast, Fig.[T2land Fig. [T3l show that we measure, with 
uncertainties smaller than ±36 degrees, the phase dif- 
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FIG. 12: The phase difference between the charged Kir P and 
S- waves. The three diagrams are the NLL scans for the S" 
(a) and B" (b) decays as well as their differences (c). The 
lines are drawn as for Fig. IIUI The data provide significant 
constraints on these angles. The four fit solutions find their 
NLL minimum at approximately the same phase differences. 



FIG. 13: The phase difference between the neutral Ktt P and 
S- waves. The three diagrams are the NLL scans for the S" 
(a) and B" (b) decays as well as their differences (c). The 
lines are drawn as for Fig. 1101 The data provide significant 
constraints on these angles. The four fit solutions find their 
NLL minimum at approximately the same phase differences. 



ferences between the Ktt S- and P-waves of the same 
charge for the B^ and B" decays. Moreover, the associ- 
ated CP-observables Scj) are measured to 35 degrees with 
neghgible systematic uncertainties (less than 10 degrees). 
An interval of order 220 degrees is excluded at the 95% 
confidence level. The charged and neutral S- and P-wave 
interference patterns thus provide sensitivity to two weak 



phases. 

The scans of the phase differences between pK and 
iiT* (892)7r show no evidence for interference at the three 
standard deviation level as shown in Fig. [Ml Here again 
the overlap in phase space between the interfering reso- 
nances is small. This contrasts with what we observe in 
the scans of the phase differences between the nonreso- 
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A^ - A(|) (degrees) 

FIG. 14: The phase difference between the pK and the 
charged Ktt P-wave. The three diagrams are the NLL scans 
for the _B° (a) and B° (b) decays as well as their differences 
(c). The lines are drawn as for Fig. 1101 The vertical scale 
cuts off Ax^ = 4, however it has been checked that all phase 
differences are consistent with the data at the three standard 
deviation level. 



TABLE VIII: The branching fractions B of B decays to quasi- 
two-body final states assuming that all K* resonances are 
isospin-1/2 states. The branching fractions of the 7^2(1430) 
and -ft'* (1680) resonances to Ktt from reference [2j| have been 
used. The upper limits at 90% confidence level, U£ are based 
on statistical uncertainties only. 



B decay final state 




B (10-'^) 


ii'*+(892)7r- 




12.6+f;;±0.9 


7^*0(892)7r° 




3.6 ±0.7 ±0.4 


iK7v);+7v'; {Kn)*+ 


^ K+tt'' 


9.4+ii+l1±l.8 


(if^)5V«; {Kn)f - 


.K+t:+ 


i.7t]i\+_\l±2.2 


p-{770)K+ 




8.01?-^ ±0.6 


N.R. 




4.4 ±0.9 ±0.5 






uc (lo-**) 


^-(1450)7^-+ 




2.1 


p-(1700)ii:+ 




1.1 


KfiU30)n° 




4.0 


K;+{U30)tv- 




16.2 


A'*"(1680)7r° 




7.5 


ii'*+(1680)7r- 
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IX. SUMMARY 

We have measured the branching fraction and CP- 
asymmetry of the B^ — ^ K tt^tt'~' decay and com- 
pared the Dahtz plots of S" -^ K+n-jr^ and B° -^ 
K~TT~^TT° using an isobar model We have extracted 
the CP-averaged isobar branching fractions and CP- 
asymmetries assuming no interference. We observe the 
B° -^ i4:*°(892)7r'' with 5.6 standard deviation signifi- 
cance. We have looked at the interference patterns in 
the Dahtz plots and put significant constraints on phase 
differences between wide intermediate states which have 
a sizable overlap in phase space. The phase shifts be- 
tween S and P-waves in the charged and neutral Kt: and 
Ktt are constrained to within ±70 degrees or less at the 
95% confidence level. Weaker constraints are observed 
for the phase shifts between the K{K)tt and nonreso- 
nant components which extend widely over the Dalitz 
plots. The phase shift differences between Ktt and Ktt 
S and P-waves are measured and found to be consistent 
with no direct CP- violation within 2 standard deviations. 
Additionally we determine the branching fraction for the 
decay B° —^ D°7r° with an accuracy comparable to that 
of the world average value of this quantity. 



nant K^tt^tt'^ and the S-waves in Fig. [T5] and Fig. [121 
We see that they are somewhat constrained. These obser- 
vations are in agreement with the fact that the fit finds 
a sizable nonresonant B'^ —> K^tt~tt^ component that 
populates the Dalitz plot far from the boundary. 
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FIG. 15: The phase difference between the nonresonant 
K'^tt'tt'^ and the charged Kn S-wave. The three diagrams 
are the NLL scans for the B° (a) and B" (b) decays as well 
as their differences (c). The lines are drawn as for Fig. 1101 
Ranges with widths of 140 (a) and 190 (b) degrees are ex- 
cluded at the 95% confidence level for the 5° and -B" decays. 
No significant difference between the B" and B" interference 
patterns is seen. 



FIG. 16: The phase difference between the nonresonant 
A"^7r~7r" and the neutral Kn S-wave. The three diagrams 
are the NLL scans for the B° (a) and S" (b) decays as well 
as their differences (c). The lines are drawn as for Fig. 1101 
Ranges with widths of 90 (a) and 110 (b) degrees are ex- 
cluded at the 95% confidence level for the B'^ and B" decays. 
No significant difference between the B" and -B" interference 
patterns is seen. 
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APPENDIX 

The four solutions of the fit are displayed in Table IIXI 
The correlation coefficients of solution-I are given in Ta- 
bles |X] and |Xll As explained in Section FVll the statistical 
uncertainty of each solution does not reflect the actual ex- 
perimental uncertainty and should not be used. The pro- 
cedure we have devised blends all four solutions and de- 
termines reliable statistical and systematic uncertainties. 
For illustration, we display the four NLL around their 
minima in Fig.[T7]for the isobar fractions and Fig. [TSl for 
the CP-asymmetries. For the neutral S-wave final states, 
the spread due to the degeneracy of the fitted fractions 
and asymmetries is quite large. 

When the NLL are far from being parabolic at their 
minima, actual scans as described at the end of SectionlVl 
are performed to derive the results. Fig. \W\ shows two 
examples of such scans for the sum of the isobar fit frac- 
tions (or the total fit fraction) and for the global CP- 
asymmetry Acp [Eq. (fT3|) ]. 
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FIG. 17: The NLL functions for the isobar fractions. The NLL functions of each solution are shown with dashed hues. The 
fitted values for the B -^ {Ktt)q^it in the four solutions are quite distinct. The envelope curves (solid lines) are used to quote 
the physical results. 
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FIG. 18: The NLL functions for the CP-asymmetries. The NLL functions of each solution are shown with dashed lines. The 
fitted values for the B -^ (A'7r)*/'7r in the four solutions are quite distinct. The envelope curves (solid lines) are used to quote 
the physical results. 



TABLE IX: Results of tlie four solutions of the fit. The fractions are the CP-averaged isobar fractions {FFj 
CP-asymmetries Acp in Section HTl [Eg. (|14|l ]. The phases cj> for the _B° decays and (f> for the P" decays are 
to B (-6°) -^ K* TT^ . The uncertainties are statistical only. They are underestimated because a parabolic 
made for the shape of the NLL close to minimum. 
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) defined with the 
measured relative 
approximation is 
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FIG. 19: NLL scans for the global CP-asymmetry (a) and the total isobar fraction (b). The scans of each solution are shown 
with dashed lines. The envelope curves (solid lines) are the scans that are used to quote the physical results. 
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TABLE X: Matrix of the correlation coefficients between the fitted parameters for the B^ Dalitz plot in solution-l. 



Variable 




(K^)*o tji 


•0(892) 


tNR 


ip(770) 0( 


K^)*+ 't>{K^)l« 4>K 


•0(892) 


(I^NR 4>p(770) 


^(K,r)* + 


100.0 
















^{K^)'o'> 


-0.9 


100.0 














iK*0(892) 


7.3 


-9.3 


100.0 












tNR 


40.6 


-27.4 


1.9 


100.0 










tp(770) 


17.3 


11.9 


8.6 


3.9 


100.0 








't'{K^r+ 


-9.1 


0.8 


0.2 


7.0 


1.3 


100.0 






<P(Kn)*o 


-31.7 


53.0 


-6.0 


-15.0 


-6.9 


32.1 100.0 






0X*O(892) 


-31.0 


50.1 


-5.5 


-14.2 


-4.9 


31.5 93.2 


100.0 




(jiNR 


-47.2 


12.9 


3.8 


-19.5 


-6.6 


54.2 61.0 


61.1 


100.0 


4>p(770) 


-35.0 


5.3 


-7.5 


-17.5 


-20.4 


31.9 44.1 


37.4 


52.4 100.0 



TABLE XI: Matrix of the correlation coefficients between the fitted parameters for the S" Dafitz plot in soIution-L 



Variable 


W)r * 


(Kn)'O ^K' 


-(892) tff 


•0(892) 


tNR 


tp(770) <t>( 


K.)r ^i 
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-1.4 
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0.9 
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20.4 
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100.0 




't>K'»{S92) 


-25.6 


52.9 


-11.7 


-6.0 


-6.6 


-12.2 


40.9 


86.9 100.0 




<Pnr 


-39.0 


6.2 


-16.2 


-0.3 


-7.5 


-11.9 


72.6 


62.3 60.3 


100.0 


<^p(770) 


-38.1 


6.2 


-18.0 


-10.3 


-4.1 


-11.8 


50.0 


53.8 47.3 


70.0 100.0 
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